{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Machine Learning in Engineering (EE658) - Spring 2021\n",
    "## Assignment #1 - Linear Regression\n",
    "### DUE DATE: Friday, February 19, 2021"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Predicting the Amount of Money Spent on Insured Customers \n",
    "\n",
    "### For this assignment, we will be analyzing insured customers' data for an insurance company. Based on a sample data that consists of the profile of insured customers, we want to be able to predict the dollar amount of money spent by the insurance company on insured customers."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Insured ustomers' Data\n",
    "The insured customers' data is in a csv file. It has information sconsisting of:\n",
    "1. age\n",
    "2. sex (female, male)\n",
    "3. bmi\n",
    "4. children\n",
    "5. smoker (yes, no)\n",
    "6. region (northeast, northwest, southeast, southwest])\n",
    "7. **expenses**\n",
    "\n",
    "The value we want to predict is **expenses**\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## (a) Import the Libraries:\n",
    "1. numpy\n",
    "2. matplotlib.pyplot\n",
    "3. pandas"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 39,
   "metadata": {},
   "outputs": [],
   "source": [
    "import numpy as np\n",
    "import matplotlib.pyplot as plt\n",
    "import pandas as pd"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## (b) Read the csv file: \"Insurance Customers Data\""
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 40,
   "metadata": {},
   "outputs": [],
   "source": [
    "df = pd.read_csv(\"insurance.csv\")"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## (c) Show a sample of the data (first 10 rows)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 41,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>age</th>\n",
       "      <th>sex</th>\n",
       "      <th>bmi</th>\n",
       "      <th>children</th>\n",
       "      <th>smoker</th>\n",
       "      <th>region</th>\n",
       "      <th>expenses</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>19</td>\n",
       "      <td>female</td>\n",
       "      <td>27.9</td>\n",
       "      <td>0</td>\n",
       "      <td>yes</td>\n",
       "      <td>southwest</td>\n",
       "      <td>16884.92</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>18</td>\n",
       "      <td>male</td>\n",
       "      <td>33.8</td>\n",
       "      <td>1</td>\n",
       "      <td>no</td>\n",
       "      <td>southeast</td>\n",
       "      <td>1725.55</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>28</td>\n",
       "      <td>male</td>\n",
       "      <td>33.0</td>\n",
       "      <td>3</td>\n",
       "      <td>no</td>\n",
       "      <td>southeast</td>\n",
       "      <td>4449.46</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>33</td>\n",
       "      <td>male</td>\n",
       "      <td>22.7</td>\n",
       "      <td>0</td>\n",
       "      <td>no</td>\n",
       "      <td>northwest</td>\n",
       "      <td>21984.47</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4</th>\n",
       "      <td>32</td>\n",
       "      <td>male</td>\n",
       "      <td>28.9</td>\n",
       "      <td>0</td>\n",
       "      <td>no</td>\n",
       "      <td>northwest</td>\n",
       "      <td>3866.86</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>5</th>\n",
       "      <td>31</td>\n",
       "      <td>female</td>\n",
       "      <td>25.7</td>\n",
       "      <td>0</td>\n",
       "      <td>no</td>\n",
       "      <td>southeast</td>\n",
       "      <td>3756.62</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>6</th>\n",
       "      <td>46</td>\n",
       "      <td>female</td>\n",
       "      <td>33.4</td>\n",
       "      <td>1</td>\n",
       "      <td>no</td>\n",
       "      <td>southeast</td>\n",
       "      <td>8240.59</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>7</th>\n",
       "      <td>37</td>\n",
       "      <td>female</td>\n",
       "      <td>27.7</td>\n",
       "      <td>3</td>\n",
       "      <td>no</td>\n",
       "      <td>northwest</td>\n",
       "      <td>7281.51</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>8</th>\n",
       "      <td>37</td>\n",
       "      <td>male</td>\n",
       "      <td>29.8</td>\n",
       "      <td>2</td>\n",
       "      <td>no</td>\n",
       "      <td>northeast</td>\n",
       "      <td>6406.41</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>9</th>\n",
       "      <td>60</td>\n",
       "      <td>female</td>\n",
       "      <td>25.8</td>\n",
       "      <td>0</td>\n",
       "      <td>no</td>\n",
       "      <td>northwest</td>\n",
       "      <td>28923.14</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "   age     sex   bmi  children smoker     region  expenses\n",
       "0   19  female  27.9         0    yes  southwest  16884.92\n",
       "1   18    male  33.8         1     no  southeast   1725.55\n",
       "2   28    male  33.0         3     no  southeast   4449.46\n",
       "3   33    male  22.7         0     no  northwest  21984.47\n",
       "4   32    male  28.9         0     no  northwest   3866.86\n",
       "5   31  female  25.7         0     no  southeast   3756.62\n",
       "6   46  female  33.4         1     no  southeast   8240.59\n",
       "7   37  female  27.7         3     no  northwest   7281.51\n",
       "8   37    male  29.8         2     no  northeast   6406.41\n",
       "9   60  female  25.8         0     no  northwest  28923.14"
      ]
     },
     "execution_count": 41,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "df.head(10)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## (d) Show some statistics about the data (describe())"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 42,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>age</th>\n",
       "      <th>bmi</th>\n",
       "      <th>children</th>\n",
       "      <th>expenses</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>count</th>\n",
       "      <td>1338.000000</td>\n",
       "      <td>1338.000000</td>\n",
       "      <td>1338.000000</td>\n",
       "      <td>1338.000000</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>mean</th>\n",
       "      <td>39.207025</td>\n",
       "      <td>30.665471</td>\n",
       "      <td>1.094918</td>\n",
       "      <td>13270.422414</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>std</th>\n",
       "      <td>14.049960</td>\n",
       "      <td>6.098382</td>\n",
       "      <td>1.205493</td>\n",
       "      <td>12110.011240</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>min</th>\n",
       "      <td>18.000000</td>\n",
       "      <td>16.000000</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>1121.870000</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>25%</th>\n",
       "      <td>27.000000</td>\n",
       "      <td>26.300000</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>4740.287500</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>50%</th>\n",
       "      <td>39.000000</td>\n",
       "      <td>30.400000</td>\n",
       "      <td>1.000000</td>\n",
       "      <td>9382.030000</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>75%</th>\n",
       "      <td>51.000000</td>\n",
       "      <td>34.700000</td>\n",
       "      <td>2.000000</td>\n",
       "      <td>16639.915000</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>max</th>\n",
       "      <td>64.000000</td>\n",
       "      <td>53.100000</td>\n",
       "      <td>5.000000</td>\n",
       "      <td>63770.430000</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "               age          bmi     children      expenses\n",
       "count  1338.000000  1338.000000  1338.000000   1338.000000\n",
       "mean     39.207025    30.665471     1.094918  13270.422414\n",
       "std      14.049960     6.098382     1.205493  12110.011240\n",
       "min      18.000000    16.000000     0.000000   1121.870000\n",
       "25%      27.000000    26.300000     0.000000   4740.287500\n",
       "50%      39.000000    30.400000     1.000000   9382.030000\n",
       "75%      51.000000    34.700000     2.000000  16639.915000\n",
       "max      64.000000    53.100000     5.000000  63770.430000"
      ]
     },
     "execution_count": 42,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "df.describe()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## (e) Show info about the data"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 43,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "<class 'pandas.core.frame.DataFrame'>\n",
      "RangeIndex: 1338 entries, 0 to 1337\n",
      "Data columns (total 7 columns):\n",
      " #   Column    Non-Null Count  Dtype  \n",
      "---  ------    --------------  -----  \n",
      " 0   age       1338 non-null   int64  \n",
      " 1   sex       1338 non-null   object \n",
      " 2   bmi       1338 non-null   float64\n",
      " 3   children  1338 non-null   int64  \n",
      " 4   smoker    1338 non-null   object \n",
      " 5   region    1338 non-null   object \n",
      " 6   expenses  1338 non-null   float64\n",
      "dtypes: float64(2), int64(2), object(3)\n",
      "memory usage: 73.3+ KB\n"
     ]
    }
   ],
   "source": [
    "df.info()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 44,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "{'northeast', 'northwest', 'southeast', 'southwest'}"
      ]
     },
     "execution_count": 44,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "set(df['region'])"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## (f) Convert columns with catgorical data into numeric values"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 45,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>age</th>\n",
       "      <th>sex</th>\n",
       "      <th>bmi</th>\n",
       "      <th>children</th>\n",
       "      <th>smoker</th>\n",
       "      <th>region</th>\n",
       "      <th>expenses</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>19</td>\n",
       "      <td>0</td>\n",
       "      <td>27.9</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>3</td>\n",
       "      <td>16884.92</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>18</td>\n",
       "      <td>1</td>\n",
       "      <td>33.8</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>2</td>\n",
       "      <td>1725.55</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>28</td>\n",
       "      <td>1</td>\n",
       "      <td>33.0</td>\n",
       "      <td>3</td>\n",
       "      <td>0</td>\n",
       "      <td>2</td>\n",
       "      <td>4449.46</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>33</td>\n",
       "      <td>1</td>\n",
       "      <td>22.7</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>21984.47</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4</th>\n",
       "      <td>32</td>\n",
       "      <td>1</td>\n",
       "      <td>28.9</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>3866.86</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "   age  sex   bmi  children  smoker  region  expenses\n",
       "0   19    0  27.9         0       1       3  16884.92\n",
       "1   18    1  33.8         1       0       2   1725.55\n",
       "2   28    1  33.0         3       0       2   4449.46\n",
       "3   33    1  22.7         0       0       1  21984.47\n",
       "4   32    1  28.9         0       0       1   3866.86"
      ]
     },
     "execution_count": 45,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "df['sex'] = df['sex'].map({'female':0, 'male':1})\n",
    "df['smoker'] = df['smoker'].map({'no':0, 'yes':1})\n",
    "df['region'] = df['region'].map({'northeast':0, 'northwest':1, 'southeast':2, 'southwest':3})\n",
    "df.head()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## (g) Based on this data, what looks to be the most correlated feature with \"Expenses\"? (Justify your answer)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 46,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>age</th>\n",
       "      <th>sex</th>\n",
       "      <th>bmi</th>\n",
       "      <th>children</th>\n",
       "      <th>smoker</th>\n",
       "      <th>region</th>\n",
       "      <th>expenses</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>region</th>\n",
       "      <td>0.002127</td>\n",
       "      <td>0.004588</td>\n",
       "      <td>0.157439</td>\n",
       "      <td>0.016569</td>\n",
       "      <td>-0.002181</td>\n",
       "      <td>1.000000</td>\n",
       "      <td>-0.006208</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>sex</th>\n",
       "      <td>-0.020856</td>\n",
       "      <td>1.000000</td>\n",
       "      <td>0.046380</td>\n",
       "      <td>0.017163</td>\n",
       "      <td>0.076185</td>\n",
       "      <td>0.004588</td>\n",
       "      <td>0.057292</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>children</th>\n",
       "      <td>0.042469</td>\n",
       "      <td>0.017163</td>\n",
       "      <td>0.012645</td>\n",
       "      <td>1.000000</td>\n",
       "      <td>0.007673</td>\n",
       "      <td>0.016569</td>\n",
       "      <td>0.067998</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>bmi</th>\n",
       "      <td>0.109341</td>\n",
       "      <td>0.046380</td>\n",
       "      <td>1.000000</td>\n",
       "      <td>0.012645</td>\n",
       "      <td>0.003968</td>\n",
       "      <td>0.157439</td>\n",
       "      <td>0.198576</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>age</th>\n",
       "      <td>1.000000</td>\n",
       "      <td>-0.020856</td>\n",
       "      <td>0.109341</td>\n",
       "      <td>0.042469</td>\n",
       "      <td>-0.025019</td>\n",
       "      <td>0.002127</td>\n",
       "      <td>0.299008</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>smoker</th>\n",
       "      <td>-0.025019</td>\n",
       "      <td>0.076185</td>\n",
       "      <td>0.003968</td>\n",
       "      <td>0.007673</td>\n",
       "      <td>1.000000</td>\n",
       "      <td>-0.002181</td>\n",
       "      <td>0.787251</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>expenses</th>\n",
       "      <td>0.299008</td>\n",
       "      <td>0.057292</td>\n",
       "      <td>0.198576</td>\n",
       "      <td>0.067998</td>\n",
       "      <td>0.787251</td>\n",
       "      <td>-0.006208</td>\n",
       "      <td>1.000000</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "               age       sex       bmi  children    smoker    region  expenses\n",
       "region    0.002127  0.004588  0.157439  0.016569 -0.002181  1.000000 -0.006208\n",
       "sex      -0.020856  1.000000  0.046380  0.017163  0.076185  0.004588  0.057292\n",
       "children  0.042469  0.017163  0.012645  1.000000  0.007673  0.016569  0.067998\n",
       "bmi       0.109341  0.046380  1.000000  0.012645  0.003968  0.157439  0.198576\n",
       "age       1.000000 -0.020856  0.109341  0.042469 -0.025019  0.002127  0.299008\n",
       "smoker   -0.025019  0.076185  0.003968  0.007673  1.000000 -0.002181  0.787251\n",
       "expenses  0.299008  0.057292  0.198576  0.067998  0.787251 -0.006208  1.000000"
      ]
     },
     "execution_count": 46,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "df.corr().sort_values('expenses')"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## (h) Refer to the customers features data by \"X\", and refer to the label feature (expenses) by \"y\" "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 47,
   "metadata": {},
   "outputs": [],
   "source": [
    "columns = set(df.columns)\n",
    "columns.remove('expenses')\n",
    "X=df[columns]\n",
    "y=df['expenses']"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 48,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>sex</th>\n",
       "      <th>smoker</th>\n",
       "      <th>children</th>\n",
       "      <th>age</th>\n",
       "      <th>region</th>\n",
       "      <th>bmi</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>19</td>\n",
       "      <td>3</td>\n",
       "      <td>27.9</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>18</td>\n",
       "      <td>2</td>\n",
       "      <td>33.8</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>3</td>\n",
       "      <td>28</td>\n",
       "      <td>2</td>\n",
       "      <td>33.0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>33</td>\n",
       "      <td>1</td>\n",
       "      <td>22.7</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4</th>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>32</td>\n",
       "      <td>1</td>\n",
       "      <td>28.9</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "   sex  smoker  children  age  region   bmi\n",
       "0    0       1         0   19       3  27.9\n",
       "1    1       0         1   18       2  33.8\n",
       "2    1       0         3   28       2  33.0\n",
       "3    1       0         0   33       1  22.7\n",
       "4    1       0         0   32       1  28.9"
      ]
     },
     "execution_count": 48,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "X.head()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 49,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "0    16884.92\n",
       "1     1725.55\n",
       "2     4449.46\n",
       "3    21984.47\n",
       "4     3866.86\n",
       "Name: expenses, dtype: float64"
      ]
     },
     "execution_count": 49,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "y.head()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## (i) Load the train_test_split function"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 50,
   "metadata": {},
   "outputs": [],
   "source": [
    "from sklearn.model_selection import train_test_split"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## (j) Split the data into **training** and **test** data sets "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 51,
   "metadata": {},
   "outputs": [],
   "source": [
    "X_train, X_test, y_train, y_test = train_test_split(X, y, test_size=0.2, random_state=42)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## (k) Import LinearRegression from sklearn.linear_model"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 52,
   "metadata": {},
   "outputs": [],
   "source": [
    "from sklearn.linear_model import LinearRegression"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## (l) Fit the model to the training data"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 53,
   "metadata": {},
   "outputs": [],
   "source": [
    "reg = LinearRegression().fit(X_train, y_train)\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## (m) Print the linear model's intercept and coefficients"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 54,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "intercept:  -11951.679263020802\n",
      "coefficients:  [-1.87048097e+01  2.36469273e+04  4.25463175e+02  2.57038298e+02\n",
      " -2.71222476e+02  3.35939380e+02]\n"
     ]
    }
   ],
   "source": [
    "print(\"intercept: \", reg.intercept_)\n",
    "print(\"coefficients: \", reg.coef_)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## (n) Use the trained model to predict the test data set"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 55,
   "metadata": {},
   "outputs": [],
   "source": [
    "y_pred = reg.predict(X_test)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## (o) Use the score function of LinearRegression to find the coefficient of determination of the prediction"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 56,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "coefficient of determination is:  0.7833214205203847\n"
     ]
    }
   ],
   "source": [
    "from sklearn.metrics import mean_absolute_error, mean_squared_error, r2_score\n",
    "print('coefficient of determination is: ', r2_score(y_test, y_pred))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## (p) Calculate: \n",
    "1. the Mean Absolute Error, \n",
    "2. Mean Squared Error, and \n",
    "3. the Root Mean Squared Error."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 57,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "MAE: 4186.940106317017\n",
      "MSE: 33639075.08997808\n",
      "RMSE: 5799.920265829357\n"
     ]
    }
   ],
   "source": [
    "print('MAE:', mean_absolute_error(y_test, y_pred))\n",
    "print('MSE:', mean_squared_error(y_test, y_pred))\n",
    "print('RMSE:', np.sqrt(mean_squared_error(y_test, y_pred)))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## (q) Show a histogram of the difference between the actual and predicted value of the test data set."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 58,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAXcAAAD4CAYAAAAXUaZHAAAAOXRFWHRTb2Z0d2FyZQBNYXRwbG90bGliIHZlcnNpb24zLjMuNCwgaHR0cHM6Ly9tYXRwbG90bGliLm9yZy8QVMy6AAAACXBIWXMAAAsTAAALEwEAmpwYAAASUUlEQVR4nO3dfYxl913f8fenXmyaQLGdHTbLrpXdwCatQaVxp66r0CjEKHECYl0pjTaqyDZYWhUMhMdgE6nmz4S2pES0QQtZskaRHdeE2kJQcIxTC6nedBwSP8bxxLHjXe3DpE4cWiQHk2//uL9tL+OZ3Zn7MDP3t++XNLrn/M45937vuUefe+Z3z0OqCklSX/7OZhcgSZo8w12SOmS4S1KHDHdJ6pDhLkkd2rbZBQBs37699uzZs9llSNJMefDBB79SVXMrTdsS4b5nzx4WFhY2uwxJmilJnlltmt0yktQhw12SOmS4S1KHDHdJ6pDhLkkdMtwlqUOGuyR1yHCXpA4Z7pLUoS1xhqrW55Wv3MPp06uemNalHTtexalTT292GdLMMNxn0CDYL6w7aJ0+nc0uQZopdstIUofOG+5JjiQ5k+SRZe0/neTzSR5N8mtD7TcnWUzyRJK3TKNoSdK5raVb5qPAbwK3nm1I8oPAfuD7q+qFJN/Z2q8EDgDfC3wX8Mkkr6mqv5l04ZKk1Z13z72q7geeW9b8E8D7q+qFNs+Z1r4fuL2qXqiqLwGLwNUTrFeStAaj9rm/BvjnSY4l+e9J/klr3wU8OzTf8db2EkkOJVlIsrC0tDRiGZKklYwa7tuAy4FrgF8C7kiyrsMZqupwVc1X1fzc3Io3EpEkjWjUcD8OfKIGPg18E9gOnACuGJpvd2uTJG2gUcP9vwI/CJDkNcDFwFeAu4EDSS5JshfYB3x6AnVKktbhvEfLJLkNeCOwPclx4BbgCHCkHR75DeBgVRXwaJI7gMeAF4EbPVJGkjZeBpm8uebn58sbZK/d4OeNzf/cNlbYCtuqtJUkebCq5lea5hmqktQhw12SOmS4S1KHDHdJ6pDhLkkdMtwlqUOGuyR1yHCXpA4Z7pLUIcNdkjpkuEtShwx3SeqQ4S5JHTLcJalDhrskdchwl6QOnTfckxxJcqbddWn5tF9IUkm2t/Ek+VCSxSQPJblqGkVLks5tLXvuHwWuW96Y5ArgzcCXh5rfyuC+qfuAQ8CHxy9RkrRe5w33qrofeG6FSR8E3svfvt/bfuDWGngAuDTJzolUKklas5H63JPsB05U1eeWTdoFPDs0fry1rfQch5IsJFlYWloapQxJ0irWHe5JXgb8CvBvx3nhqjpcVfNVNT83NzfOU0mSltk2wjLfDewFPpcEYDfwmSRXAyeAK4bm3d3aJEkbaN177lX1cFV9Z1Xtqao9DLperqqqU8DdwLvaUTPXAM9X1cnJlixJOp+1HAp5G/A/gNcmOZ7khnPM/kfAU8Ai8NvAT06kSknSupy3W6aq3nme6XuGhgu4cfyyJEnj8AxVSeqQ4S5JHTLcJalDhrskdchwl6QOGe6S1CHDXZI6ZLhLUocMd0nqkOEuSR0y3CWpQ4a7JHXIcJekDhnuktQhw12SOmS4S1KH1nInpiNJziR5ZKjt3yX5fJKHkvxBkkuHpt2cZDHJE0neMqW6JUnnsJY9948C1y1ruwf4vqr6h8AXgJsBklwJHAC+ty3zn5NcNLFqJUlrct5wr6r7geeWtf1pVb3YRh8Adrfh/cDtVfVCVX2Jwb1Ur55gvZKkNZhEn/uPA3/chncBzw5NO97aXiLJoSQLSRaWlpYmUIYk6ayxwj3J+4AXgY+td9mqOlxV81U1Pzc3N04ZkqRlto26YJJ/DfwIcG1VVWs+AVwxNNvu1iZJ2kAj7bknuQ54L/CjVfVXQ5PuBg4kuSTJXmAf8Onxy5Qkrcd599yT3Aa8Edie5DhwC4OjYy4B7kkC8EBV/ZuqejTJHcBjDLprbqyqv5lW8ZKkleX/96hsnvn5+VpYWNjsMmbG4At18z+3jRW2wrYqbSVJHqyq+ZWmeYaqJHXIcJekDhnuktQhw12SOmS4S1KHDHdJ6pDhLkkdMtwlqUOGuyR1yHCXpA4Z7pLUIcNdkjpkuEtShwx3SeqQ4S5JHTLcJalD5w33JEeSnEnyyFDb5UnuSfJke7ystSfJh5IsJnkoyVXTLF6StLK17Ll/FLhuWdtNwL1VtQ+4t40DvJXBfVP3AYeAD0+mTEnSepw33KvqfuC5Zc37gaNt+Chw/VD7rTXwAHBpkp0TqlWStEaj9rnvqKqTbfgUsKMN7wKeHZrveGt7iSSHkiwkWVhaWhqxDEnSSsb+QbUGdy1e952Lq+pwVc1X1fzc3Ny4ZUiShowa7qfPdre0xzOt/QRwxdB8u1ubJGkDjRrudwMH2/BB4K6h9ne1o2auAZ4f6r6RJG2QbeebIcltwBuB7UmOA7cA7wfuSHID8Azwjjb7HwFvAxaBvwLePYWaJUnncd5wr6p3rjLp2hXmLeDGcYuSJI3HM1QlqUOGuyR1yHCXpA4Z7pLUIcNdkjpkuEtShwx3SeqQ4S5JHTLcJalDhrskdchwl6QOGe6S1CHDXZI6ZLhLUocMd0nqkOEuSR0aK9yT/FySR5M8kuS2JN+aZG+SY0kWk3w8ycWTKlaStDYjh3uSXcDPAPNV9X3ARcAB4APAB6vqe4CvAjdMolBJ0tqN2y2zDfi7SbYBLwNOAm8C7mzTjwLXj/kakqR1Gjncq+oE8O+BLzMI9eeBB4GvVdWLbbbjwK6Vlk9yKMlCkoWlpaVRy5AkrWCcbpnLgP3AXuC7gJcD1611+ao6XFXzVTU/Nzc3ahmSpBWM0y3zQ8CXqmqpqv4a+ATweuDS1k0DsBs4MWaNkqR1Gifcvwxck+RlSQJcCzwG3Ae8vc1zELhrvBIlSes1Tp/7MQY/nH4GeLg912Hgl4GfT7IIvAL4yATqlCStw7bzz7K6qroFuGVZ81PA1eM8ryRpPJ6hKkkdMtwlqUOGuyR1yHCXpA4Z7pLUIcNdkjpkuEtShwx3SeqQ4S5JHTLcJalDhrskdchwl6QOGe6S1CHDXZI6ZLhLUocMd0nq0FjhnuTSJHcm+XySx5P8sySXJ7knyZPt8bJJFStJWptx99x/A/hvVfX3ge8HHgduAu6tqn3AvW1ckrSBRg73JN8BvIF2j9Sq+kZVfQ3YDxxtsx0Frh+vREnSeo2z574XWAJ+N8lfJPmdJC8HdlTVyTbPKWDHSgsnOZRkIcnC0tLSGGVIkpYbJ9y3AVcBH66q1wH/h2VdMFVVQK20cFUdrqr5qpqfm5sbowxJ0nLjhPtx4HhVHWvjdzII+9NJdgK0xzPjlShJWq+Rw72qTgHPJnlta7oWeAy4GzjY2g4Cd41VoSRp3baNufxPAx9LcjHwFPBuBl8YdyS5AXgGeMeYryFJWqexwr2qPgvMrzDp2nGeV5I0Hs9QlaQOGe6S1CHDXZI6ZLhLUocMd0nqkOEuSR0a9zj3TffKV+7h9OlnNrsMSdpSZj7cB8G+4uVrOpbNLkDSFme3jCR1yHCXpA4Z7pLUIcNdkjpkuEtShwx3SeqQ4S5JHTLcJalDY4d7kouS/EWSP2zje5McS7KY5OPtLk2SpA00iT339wCPD41/APhgVX0P8FXghgm8hiRpHcYK9yS7gR8GfqeNB3gTcGeb5Shw/TivIUlav3H33P8j8F7gm238FcDXqurFNn4c2DXma0iS1mnkcE/yI8CZqnpwxOUPJVlIsrC0tDRqGZKkFYyz5/564EeTPA3czqA75jeAS5OcvdrkbuDESgtX1eGqmq+q+bm5uTHKkCQtN3K4V9XNVbW7qvYAB4A/q6p/BdwHvL3NdhC4a+wqJUnrMo3j3H8Z+Pkkiwz64D8yhdeQJJ3DRG7WUVWfAj7Vhp8Crp7E80qSRuMZqpLUoZm/zZ4uFJcwOI3iwrFjx6s4derpzS5DM8pw14x4gQvtXrmnT19YX2aaLLtlJKlDhrskdchwl6QOGe6S1CHDXZI6ZLhLUocMd0nqkOEuSR0y3CWpQ4a7JHXIcJekDhnuktQhw12SOmS4S1KHRg73JFckuS/JY0keTfKe1n55knuSPNkeL5tcuZKktRhnz/1F4Beq6krgGuDGJFcCNwH3VtU+4N42LknaQCOHe1WdrKrPtOG/BB4HdgH7gaNttqPA9WPWKElap4n0uSfZA7wOOAbsqKqTbdIpYMcqyxxKspBkYWlpaRJlSJKascM9ybcBvw/8bFV9fXhaVRWr3Butqg5X1XxVzc/NzY1bhiRpyFjhnuRbGAT7x6rqE635dJKdbfpO4Mx4JUqS1muco2UCfAR4vKp+fWjS3cDBNnwQuGv08iRJo9g2xrKvB34MeDjJZ1vbrwDvB+5IcgPwDPCOsSqUJK3byOFeVX8OZJXJ1476vJKk8XmGqiR1yHCXpA4Z7pLUIcNdkjo0ztEykqbqEgZHHF84dux4FadOPb3ZZXTBcJe2rBdY5QTvbp0+fWF9mU2T3TKS1CHDXZI6ZLhLUocMd0nqkOEuSR0y3CWpQ4a7JHXI49wlbSGeuDUphrukLcQTtybFbhlJ6tDUwj3JdUmeSLKY5KZpvY4k6aWmEu5JLgL+E/BW4ErgnUmunMZrSZJealp77lcDi1X1VFV9A7gd2D+l15IkLTOtH1R3Ac8OjR8H/unwDEkOAYfa6P9O8sToL7fiDxLbga+M/pybYh01b6kjCjZoXU/0Pc/I9vGS9zwjdb/ELG7bG7auxzhC6FWrTdi0o2Wq6jBweFrPn2Shquan9fzTMIs1w2zWPYs1g3VvpFmsedi0umVOAFcMje9ubZKkDTCtcP+fwL4ke5NcDBwA7p7Sa0mSlplKt0xVvZjkp4A/AS4CjlTVo9N4rXOYWpfPFM1izTCbdc9izWDdG2kWa/5/UnVhnQ0mSRcCz1CVpA4Z7pLUoZkJ9yT/MsmjSb6ZZH7ZtJvbZQ6eSPKWofYVL4HQfug91to/3n70JcklbXyxTd8z4ffwq0lOJPls+3vbpN/DRtqKl5hI8nSSh9v6XWhtlye5J8mT7fGy1p4kH2r1P5TkqqHnOdjmfzLJwQnXeCTJmSSPDLVNrMYk/7itg8W27EQOHF+l7i29TSe5Isl9SR5r+fGe1r7l1/fYqmom/oB/ALwW+BQwP9R+JfA54BJgL/BFBj/iXtSGXw1c3Oa5si1zB3CgDf8W8BNt+CeB32rDB4CPT/g9/Crwiyu0T+w9bODnsWptm7ydPA1sX9b2a8BNbfgm4ANt+G3AHzM4a+Ya4Fhrvxx4qj1e1oYvm2CNbwCuAh6ZRo3Ap9u8acu+dYp1b+ltGtgJXNWGvx34Qqtty6/vcf9mZs+9qh6vqpXOYt0P3F5VL1TVl4BFBpc/WPESCO1b9U3AnW35o8D1Q891tA3fCVy7Qd/Ck3wPG2WWLjEx/Lku/7xvrYEHgEuT7ATeAtxTVc9V1VeBe4DrJlVMVd0PPDeNGtu0v1dVD9QgeW5lQtvGKnWvZkts01V1sqo+04b/EnicwRn0W359j2tmwv0cVrrUwa5ztL8C+FpVvbis/W89V5v+fJt/kn6q/bt35Oy/ghN+Dxtltdo2WwF/muTBDC5xAbCjqk624VPAjja83vU+TZOqcVcbXt4+TTOxTWfQzfo64Bizvb7XZEuFe5JPJnlkhb+tukf4Eud5Dx8Gvhv4R8BJ4D9sZq2d+oGquorBFUlvTPKG4Ylt72pLH/87CzUOmYltOsm3Ab8P/GxVfX142oyt7zXbUndiqqofGmGxc13qYKX2/8XgX61tbS9heP6zz3U8yTbgO9r8a7bW95Dkt4E/nMJ72Chb8hITVXWiPZ5J8gcMugFOJ9lZVSfbv9Fn2uyrvYcTwBuXtX9qyqVPqsYTbXj5/FNRVafPDm/VbTrJtzAI9o9V1Sda80yu7/XYUnvuI7obOJDBkS57gX0MfuBY8RII7Vv6PuDtbfmDwF1Dz3X2V/C3A3/W5p+IthGd9S+As0cdTPI9bJQtd4mJJC9P8u1nh4E3M1jHw5/r8s/7Xe0IiWuA59u/6n8CvDnJZa2b4c2tbZomUmOb9vUk17R+7HcxxW1jq2/TbR18BHi8qn59aNJMru912exfdNf6x2DDOc7gJounGazYs9Pex+AX+CcY+qWawS/fX2jT3jfU/moGG9oi8F+AS1r7t7bxxTb91RN+D78HPAw8xGAj2jnp97DBn8mKtW3iNvJqBkdffA549GxNDPpz7wWeBD4JXN7aw+CmMl9sn8vwUVg/3tbtIvDuCdd5G4MujL9u2/QNk6wRmGcQsl8EfpN2JvqU6t7S2zTwAwy6XB4CPtv+3jYL63vcPy8/IEkd6qFbRpK0jOEuSR0y3CWpQ4a7JHXIcJekDhnuktQhw12SOvR/AS+trSm4ffd+AAAAAElFTkSuQmCC\n",
      "text/plain": [
       "<Figure size 432x288 with 1 Axes>"
      ]
     },
     "metadata": {
      "needs_background": "light"
     },
     "output_type": "display_data"
    }
   ],
   "source": [
    "pred = y_pred.reshape(1,y_pred.shape[0])\n",
    "y_true = np.array(y_test).reshape(1,y_test.shape[0])\n",
    "plt.hist(y_true[0]-pred[0],color='blue', edgecolor='black',bins=5)\n",
    "plt.show()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 59,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>sex</th>\n",
       "      <th>smoker</th>\n",
       "      <th>children</th>\n",
       "      <th>age</th>\n",
       "      <th>region</th>\n",
       "      <th>bmi</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>560</th>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>2</td>\n",
       "      <td>46</td>\n",
       "      <td>1</td>\n",
       "      <td>20.0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1285</th>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>47</td>\n",
       "      <td>0</td>\n",
       "      <td>24.3</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1142</th>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>52</td>\n",
       "      <td>2</td>\n",
       "      <td>24.9</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>969</th>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>5</td>\n",
       "      <td>39</td>\n",
       "      <td>2</td>\n",
       "      <td>34.3</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>486</th>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>3</td>\n",
       "      <td>54</td>\n",
       "      <td>1</td>\n",
       "      <td>21.5</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>...</th>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1095</th>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>4</td>\n",
       "      <td>18</td>\n",
       "      <td>0</td>\n",
       "      <td>31.4</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1130</th>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>5</td>\n",
       "      <td>39</td>\n",
       "      <td>2</td>\n",
       "      <td>23.9</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1294</th>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>58</td>\n",
       "      <td>0</td>\n",
       "      <td>25.2</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>860</th>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>2</td>\n",
       "      <td>37</td>\n",
       "      <td>3</td>\n",
       "      <td>47.6</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1126</th>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>55</td>\n",
       "      <td>3</td>\n",
       "      <td>29.9</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "<p>1070 rows × 6 columns</p>\n",
       "</div>"
      ],
      "text/plain": [
       "      sex  smoker  children  age  region   bmi\n",
       "560     0       0         2   46       1  20.0\n",
       "1285    0       0         0   47       0  24.3\n",
       "1142    0       0         0   52       2  24.9\n",
       "969     0       0         5   39       2  34.3\n",
       "486     0       0         3   54       1  21.5\n",
       "...   ...     ...       ...  ...     ...   ...\n",
       "1095    0       0         4   18       0  31.4\n",
       "1130    0       0         5   39       2  23.9\n",
       "1294    1       0         0   58       0  25.2\n",
       "860     0       1         2   37       3  47.6\n",
       "1126    1       0         0   55       3  29.9\n",
       "\n",
       "[1070 rows x 6 columns]"
      ]
     },
     "execution_count": 59,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "X_train"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## (r) Explain the meaning behind the coefficients of the model"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "collapsed": true
   },
   "source": [
    "### As shown in (m), first two columns are positive float,  and the corresponding features are smoker and children   have positive effect on expenses, which means person who smoking and have more children will have more expenses."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 60,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "560      9193.84\n",
       "1285     8534.67\n",
       "1142    27117.99\n",
       "969      8596.83\n",
       "486     12475.35\n",
       "          ...   \n",
       "1095     4561.19\n",
       "1130     8582.30\n",
       "1294    11931.13\n",
       "860     46113.51\n",
       "1126    10214.64\n",
       "Name: expenses, Length: 1070, dtype: float64"
      ]
     },
     "execution_count": 60,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "y_train"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## (s) Use your own implementation of the Batch Gradient Descent to find the intercept and coefficients of the linear regression model"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 61,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "coefficients:  [[ 32.86856902]\n",
      " [379.81596159]\n",
      " [ 72.64526861]\n",
      " [220.85604174]\n",
      " [-35.76413969]\n",
      " [293.12838307]]\n",
      "intercept:  -4402.016202366106\n",
      "MAE: 9113.435438884639\n",
      "MSE: 128575514.48980838\n",
      "RMSE: 11339.114360910573\n"
     ]
    }
   ],
   "source": [
    "learning_rate = 0.0001\n",
    "m = X_train.shape[0]\n",
    "W = np.random.randn(X_train.shape[1], 1)\n",
    "bias = 0\n",
    "n_iterations=1000\n",
    "for iteration in range(n_iterations):\n",
    "    y_hat = np.dot(X_train, W) + bias\n",
    "    dw = (1/m) * np.dot(X_train.T,(y_hat-np.array(y_train).reshape(-1,1)))\n",
    "    db = (1/m) * np.sum(y_hat-np.array(y_train))\n",
    "    W -= learning_rate * dw\n",
    "    bias -= learning_rate * db\n",
    "print(\"coefficients: \",W)\n",
    "print(\"intercept: \",bias)\n",
    "y_pred_test = X_test.dot(W) + bias\n",
    "print('MAE:', mean_absolute_error(y_test, y_pred_test))\n",
    "print('MSE:', mean_squared_error(y_test, y_pred_test))\n",
    "print('RMSE:', np.sqrt(mean_squared_error(y_test, y_pred_test)))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Lower learning rate"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 62,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "coefficients:  [[ 62.6016056 ]\n",
      " [757.74523943]\n",
      " [138.30813512]\n",
      " [223.5219775 ]\n",
      " [-71.97799649]\n",
      " [326.33250504]]\n",
      "intercept:  -5652.0650685815135\n",
      "MAE: 8981.903734644924\n",
      "MSE: 125292314.44084659\n",
      "RMSE: 11193.404952955405\n"
     ]
    }
   ],
   "source": [
    "learning_rate = 0.0002\n",
    "m = X_train.shape[0]\n",
    "W = np.random.randn(X_train.shape[1], 1)\n",
    "bias = 0\n",
    "n_iterations=1000\n",
    "for iteration in range(n_iterations):\n",
    "    y_hat = np.dot(X_train, W) + bias\n",
    "    dw = (1/m) * np.dot(X_train.T,(y_hat-np.array(y_train).reshape(-1,1)))\n",
    "    db = (1/m) * np.sum(y_hat-np.array(y_train))\n",
    "    W -= learning_rate * dw\n",
    "    bias -= learning_rate * db\n",
    "print(\"coefficients: \",W)\n",
    "print(\"intercept: \",bias)\n",
    "y_pred_test = X_test.dot(W) + bias\n",
    "print('MAE:', mean_absolute_error(y_test, y_pred_test))\n",
    "print('MSE:', mean_squared_error(y_test, y_pred_test))\n",
    "print('RMSE:', np.sqrt(mean_squared_error(y_test, y_pred_test)))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## bigger learning rate"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 63,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "coefficients:  [[  92.83571414]\n",
      " [1126.60983532]\n",
      " [ 196.36599623]\n",
      " [ 224.15159946]\n",
      " [-104.02272367]\n",
      " [ 332.29624543]]\n",
      "intercept:  -5969.39657740602\n",
      "MAE: 8862.470595308701\n",
      "MSE: 122354161.39100724\n",
      "RMSE: 11061.38153175304\n"
     ]
    }
   ],
   "source": [
    "learning_rate = 0.0003\n",
    "m = X_train.shape[0]\n",
    "W = np.random.randn(X_train.shape[1], 1)\n",
    "bias = 0\n",
    "n_iterations=1000\n",
    "for iteration in range(n_iterations):\n",
    "    y_hat = np.dot(X_train, W) + bias\n",
    "    dw = (1/m) * np.dot(X_train.T,(y_hat-np.array(y_train).reshape(-1,1)))\n",
    "    db = (1/m) * np.sum(y_hat-np.array(y_train))\n",
    "    W -= learning_rate * dw\n",
    "    bias -= learning_rate * db\n",
    "print(\"coefficients: \",W)\n",
    "print(\"intercept: \",bias)\n",
    "y_pred_test = X_test.dot(W) + bias\n",
    "print('MAE:', mean_absolute_error(y_test, y_pred_test))\n",
    "print('MSE:', mean_squared_error(y_test, y_pred_test))\n",
    "print('RMSE:', np.sqrt(mean_squared_error(y_test, y_pred_test)))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## (t)\tDemonstrate the effect of using different Learning Rate Parameter."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### If learning rate parameter is too small, the algorithm will take more iterations to converge, and if the learning rate is too high, it will jump over the valley and end up on the other side, unfortunately, you will miss the minimum cost value and make the algorithm diverge."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## (u)\tCompare your results from (m) and (s)."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 64,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "MAE: 8862.470595308701\n",
      "MSE: 122354161.39100724\n",
      "RMSE: 11061.38153175304\n"
     ]
    }
   ],
   "source": [
    "y_pred_test = X_test.dot(W) + bias\n",
    "print('MAE:', mean_absolute_error(y_test, y_pred_test))\n",
    "print('MSE:', mean_squared_error(y_test, y_pred_test))\n",
    "print('RMSE:', np.sqrt(mean_squared_error(y_test, y_pred_test)))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## (v)\tSubstitute the Stochastic Gradient Descent for the BGD. Compare results with Question (s)."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 65,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "coefficients:  [[  1.63393041]\n",
      " [  1.5174983 ]\n",
      " [  3.67954369]\n",
      " [115.99069751]\n",
      " [  1.45253882]\n",
      " [ 86.56536815]]\n",
      "intercept:  2.68413726586424\n"
     ]
    }
   ],
   "source": [
    "learning_rate = 0.0003\n",
    "m = X_train.shape[0]\n",
    "W = np.random.randn(X_train.shape[1], 1)\n",
    "bias = 0\n",
    "n_iterations=1000\n",
    "for iteration in range(n_iterations):\n",
    "    random_index = np.random.randint(m)\n",
    "    xi = X_train[random_index: random_index+1]\n",
    "    yi = y_train[random_index: random_index+1]\n",
    "    y_hat = np.dot(xi, W) + bias\n",
    "    dw = (1/m) * np.dot(xi.T,(y_hat-np.array(yi).reshape(-1,1)))\n",
    "    db = (1/m) * np.sum(y_hat-np.array(yi))\n",
    "    W -= learning_rate * dw\n",
    "    bias -= learning_rate * db\n",
    "print(\"coefficients: \",W)\n",
    "print(\"intercept: \",bias)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 66,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "MAE: 7623.863697339143\n",
      "MSE: 174291712.98593214\n",
      "RMSE: 13201.9586799055\n"
     ]
    }
   ],
   "source": [
    "y_pred_test = X_test.dot(W) + bias\n",
    "print('MAE:', mean_absolute_error(y_test, y_pred_test))\n",
    "print('MSE:', mean_squared_error(y_test, y_pred_test))\n",
    "print('RMSE:', np.sqrt(mean_squared_error(y_test, y_pred_test)))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# (w)\tUse the Mini Batch Gradient Descent and compare with (s) and (v)."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 67,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "coefficients:  [[  76.84326464]\n",
      " [1116.98702564]\n",
      " [ 161.97885431]\n",
      " [ 197.68541627]\n",
      " [-125.51097766]\n",
      " [ 203.58196409]]\n",
      "intercept:  -821.6819999815285\n"
     ]
    }
   ],
   "source": [
    "learning_rate = 0.0003\n",
    "m = X_train.shape[0]\n",
    "minibatch_size = 20\n",
    "W = np.random.randn(X_train.shape[1], 1)\n",
    "bias = 0\n",
    "n_iterations=1000\n",
    "shuffled_indices = np.random.permutation(m)\n",
    "X_b_shuffled = np.array(X_train)[shuffled_indices]\n",
    "y_shuffled = np.array(y_train)[shuffled_indices]\n",
    "for iteration in range(n_iterations):\n",
    "    for i in range(0, m, minibatch_size):\n",
    "        random_index = np.random.randint(m)\n",
    "        xi = X_b_shuffled[i:i+minibatch_size]\n",
    "        yi = y_shuffled[i:i+minibatch_size]\n",
    "        y_hat = np.dot(xi, W) + bias\n",
    "        dw = (1/m) * np.dot(xi.T,(y_hat-np.array(yi).reshape(-1,1)))\n",
    "        db = (1/m) * np.sum(y_hat-np.array(yi))\n",
    "        W -= learning_rate * dw\n",
    "        bias -= learning_rate * db\n",
    "print(\"coefficients: \",W)\n",
    "print(\"intercept: \",bias)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 68,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "MAE: 8943.573653770829\n",
      "MSE: 124550197.87735577\n",
      "RMSE: 11160.205996188231\n"
     ]
    }
   ],
   "source": [
    "y_pred_test = X_test.dot(W) + bias\n",
    "print('MAE:', mean_absolute_error(y_test, y_pred_test))\n",
    "print('MSE:', mean_squared_error(y_test, y_pred_test))\n",
    "print('RMSE:', np.sqrt(mean_squared_error(y_test, y_pred_test)))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## (x)\tPerform Data Normalization and repeat question (s)."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 69,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "coefficients:  [[ 63.43645904]\n",
      " [892.87950677]\n",
      " [ 79.18689404]\n",
      " [322.45812796]\n",
      " [ -1.92287724]\n",
      " [221.84299542]]\n",
      "intercept:  13342.895447445331\n",
      "MAE: 8877.83022805174\n",
      "MSE: 134096966.45878394\n",
      "RMSE: 11580.024458470885\n"
     ]
    }
   ],
   "source": [
    "from sklearn.preprocessing import StandardScaler\n",
    "scaler = StandardScaler()\n",
    "X = scaler.fit_transform(X)\n",
    "X_train, X_test, y_train, y_test = train_test_split(X, y, test_size=0.2, random_state=42)\n",
    "learning_rate = 0.0001\n",
    "m = X_train.shape[0]\n",
    "W = np.random.randn(X_train.shape[1], 1)\n",
    "bias = 0\n",
    "n_iterations=1000\n",
    "for iteration in range(n_iterations):\n",
    "    y_hat = np.dot(X_train, W) + bias\n",
    "    dw = (1/m) * np.dot(X_train.T,(y_hat-np.array(y_train).reshape(-1,1)))\n",
    "    db = (1/m) * np.sum(y_hat-np.array(y_train))\n",
    "    W -= learning_rate * dw\n",
    "    bias -= learning_rate * db\n",
    "print(\"coefficients: \",W)\n",
    "print(\"intercept: \",bias)\n",
    "y_pred_test = X_test.dot(W) + bias\n",
    "print('MAE:', mean_absolute_error(y_test, y_pred_test))\n",
    "print('MSE:', mean_squared_error(y_test, y_pred_test))\n",
    "print('RMSE:', np.sqrt(mean_squared_error(y_test, y_pred_test)))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## (y)\tUse the Normal Equation to solve the linear regression problem."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 77,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "MAE: 4186.940106317017\n",
      "MSE: 33639075.089978084\n",
      "RMSE: 5799.920265829357\n"
     ]
    }
   ],
   "source": [
    "X_b = np.c_[np.ones((X_train.shape[0], 1)), X_train]\n",
    "W = np.linalg.inv(X_b.T.dot(X_b)).dot(X_b.T).dot(y_train)\n",
    "X_b_test = np.c_[np.ones((X_test.shape[0], 1)),X_test]\n",
    "y_pred_test = X_b_test.dot(W)\n",
    "print('MAE:', mean_absolute_error(y_test, y_pred_test))\n",
    "print('MSE:', mean_squared_error(y_test, y_pred_test))\n",
    "print('RMSE:', np.sqrt(mean_squared_error(y_test, y_pred_test)))"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": []
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 3",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.6.5"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 2
}
